The behavior of Large Language Models (LLMs) when facing contextual information that conflicts with their internal parametric knowledge is inconsistent, with no generally accepted explanation for the expected outcome distribution. Recent work has identified in autoregressive transformer models a class of neurons -- called entropy neurons -- that produce a significant effect on the model output entropy while having an overall moderate impact on the ranking of the predicted tokens. In this paper, we investigate the preliminary claim that these neurons are involved in inhibiting context copying behavior in transformers by looking at their role in resolving conflicts between contextual and parametric information. We show that entropy neurons are responsible for suppressing context copying across a range of LLMs, and that ablating them leads to a significant change in the generation process. These results enhance our understanding of the internal dynamics of LLMs when handling conflicting information.

Large Language Models (LLMs) often encounter conflicts between their learned, internal (parametric knowledge, PK) and external knowledge provided during inference (contextual knowledge, CK). Understanding how LLMs models prioritize one knowledge source over the other remains a challenge. In this paper, we propose a novel probing framework to explore the mechanisms governing the selection between PK and CK in LLMs. Using controlled prompts designed to contradict the model's PK, we demonstrate that specific model activations are indicative of the knowledge source employed. We evaluate this framework on various LLMs of different sizes and demonstrate that mid-layer activations, particularly those related to relations in the input, are crucial in predicting knowledge source selection, paving the way for more reliable models capable of handling knowledge conflicts effectively.

The takeover of a federal wildlife refuge in Oregon by anti-government protesters wasn't violent, a county sheriff testified, but it was far from peaceful. "Certainly it's not normal to have a hundred people walking around with firearms on our streets," Harney County Sheriff Dave Ward said Wednesday, becoming the first witness to testify in the trial of Ammon Bundy, his brother Ryan and five others charged with conspiracy in the 41-day takeover of a federal wildlife preserve in southeast Oregon. Ward testified in the federal courthouse here that features a wall engraved with a quote from Thomas Jefferson: "The boisterous sea of liberty is never without a wave." The sheriff told the jury how such a wave, ridden by Ammon Bundy and his supporters, came crashing down on Harney County early this year. On Jan. 2, Ward was watching what he thought was a live TV feed of a Bundy-led rally just three blocks from his office in Burns.